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Abstract 

Direct instruction to students enrolled in a computer literacy program at the 
undergraduate level frequently involves difficulties due to varied knowledge levels 
and skills among the students, as well as an increase in the number of unmotivated 
students. An available solution is the pair problem solving approach which can prove 
to be effective as an effective method. This report shares the findings of an 
investigation regarding the efficacy of pair problem solving, as compared to 
individual problem solving in computer literacy education. Furthermore, the paired 
approach analysis was able to extract specific criteria for successful pairs. The 
research, which included two (paired and individual) 15-minute practical 
examinations and questionnaires, a test on basic scholastic ability, and a survey on PC 
experiences, was conducted with approximately 280 students from three universities 
who were enrolled in a computer literacy program in 2008 and 2009. The results 
reveal that the overall scores of the pairs exceeded those of the individuals. Moreover, 
more than 90% of students found pair problem solving to be a positive experience. 
From the viewpoint of learning effectiveness, it is worth mentioning that the most 
effective pair combinations included those with a small difference in basic academic 
ability, a large difference in PC experience, and a partner of the opposite sex. 
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Introduction 


With the advent of declining university enrollments, university instructions are 
becoming difficult to be followed because of different cognitive and behavioral 
characteristics observed in students, such as lower academic ability and intellectual 
curiosity (Figure 1). 

The skills needed to operate a computer have diversified and the computer literacy 
gap has expanded. 

Because of this, there have been arguments for the necessity to strictly review 
educational content and methodology particularly for computer literacy education 
(Murakami et ah, 2008). Given the current situation, interactive and participatory 
approaches for effective instructions that focus on the student have been taking place. 
It has been reported that cooperative learning is very effective in research and in 
practice, particularly for pairs and small groups. Therefore, the expectations from 
these methods are increasing (Yasunaga, 2008, Tachibana et ah, 2010). 



Figure 1 Background of research 
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The effects of the pair approach within information education suggest possibilities, 
such as encouraging infonnation literacy, and stimulating students’ desire to learn, 
(Takahashi et ah, 2004) as well as improving their ability to complete tasks, solve 
problems, and learn independently (Terakawa et ah, 2005). On the other hand, there 
are indications that depending on the pair combination, there may not always be an 
effect on learning or that there might be issues with developing methods to form 
effective pairs (Kaneko et ah, 2007, Takahashi et ah, 2010). However, regardless of 
the numerous reports on the subject, there is a lack of understanding of pair 
combinations or combination criteria because there are few studies that deal with this 
issue. Keeping this in mind, the authors of this study introduced a pair approach into 
university computer literacy education in 2008. They examined the effectiveness of 
this approach by comparing individual problem solving with pair cooperative problem 
solving and verifying the effects pair combinations have on the results. Thus far, it is 
evident that pair cooperative problem solving improved the overall task achievement 
level and was particularly effective for students with lower grades and with mixed- 
gender pairs (Uchida et al., 2010). The students’ assessment of pair learning was high, 
indicating that this method was effective in meeting students’ needs (Uchida et al., 
2010). However, this method also has certain disadvantages such as striking 
differences observed between pair results and either no or negative effects with 
certain pair combinations. 

This study first reports the problem solving results with pairs from a pair combination 
criteria perspective based on the results of pair solving approach in class, conducted 
from 2008 to 2009. It also focuses on the problem-solving process as an index for 
learners’ awareness toward working as pairs as well as the quantitative changes in 
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utterances among pairs, as a means to examine the issues of problem solving for 
selected pairs. Finally, the study considers the pair learning effect from the amount of 
utterances and survey results to determine how cooperative problem solving is 
effective through conversation and student trends. 


Methodology 

The subjects of this study were enrolled in a computer literacy program in 3 
departments of 2 private universities in Aichi Prefecture. A total of 7 classes and 280 
students participated each year for 2008 and 2009. In April, students were surveyed 
on pair combination criteria and in July, experimental classes were held for pair 
testing (Figure 2). 


April 


Survey pair composition 
basics 


Classes using traditional 
April -June approach (one teacher 

teaches all students at once) 


July Test classes for pair work 



Figure 2 Outline of the study 
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Pair Combination Criteria 

In 2008, students were surveyed on their basic academic ability, computer experience, 
interest in computers, and typing speed in order to gain basic data regarding the pair 
combination criteria. Of these four criteria, a prior study has acknowledged the 
relationship between basic academic ability and scholastic perfonnance of students 
after enrolling in university, adapting to university education, and scores in the 
national exams. Three other items reflected computer literacy before university, 
which is the basic premise for computer literacy education, and were included 
because objective data on them is relatively easy to obtain. 

Given the results of 2008, the 2009 survey focused on 2 indicators; basic academic 
ability, which implied involvement in problem solving and performance in pairs, and 
computer experience before university. 

The basic academic ability survey consisted of 20 math and kanji (Japanese character) 
problems and used an adjusted difficulty level so that perfonnance would 
approximate a normal distribution. Math problems were composed of basic math 
problems developed to measure university students’ academic abilities. Kanji 
problems referenced the kanji test that measures basic Japanese ability. The survey 
lasted 20 minutes and surveys were collected individually for each participant. 

The survey on computer experience before university had 20 multiple-choice 
questions about the Internet, software, and computer usage inside and outside the 
school. In the 2008 survey, there were few questions and the multiple choice answers 
varied based on the question. The 2009 survey improved on these two issues. The 
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survey time lasted 5 minutes and surveys were collected individually for each 
participant. 

Pair Problem Solving 

After 8-10 practical computer literacy classes, students were tested (Test 1 and Test 
2) individually and in pairs for 15 minutes (22 questions) based on word-processing 
proficiency. Pair groupings were randomly selected to determine the effect of pair 
combination criteria. Then, students in each department were divided without bias per 
class. Approximately half of the students took Test 1 individually followed by Test 2 
in pairs. The remainder of the class took Test 1 in pairs followed by Test 2 
individually. In each of the divided groups, almost all students were in the same year 
of school and from the same academic discipline. Since one teacher taught the same 
material to both groups, the difference between the groups is presumed to be 
negligible. During the test, students solved problems in pairs and individually, and the 
results were collected individually for each participant. According to preliminary 
investigation, the dispersion for Test 1 and Test 2 was set to a certain level adjusting 
the difficulty level so that the average variance of correct responses differed by 15 to 
20 percent. Furthermore, in order to eliminate the issues with testing order in 2009 
and 2008, the tests were conducted in the reverse order (switching Test 1 and Test 2). 

Before the pair test, students were given five minutes for free conversation to develop 
smooth communication for each pair’s first encounter. Twenty minutes of 
conversation was recorded from the time free conversation began to the end of the 
pair test. After the test, the students took a survey about their method of problem 
solving in pairs. The 2009 survey improved upon the issues with multiple choice 
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expressions that were apparent during the 2008 study. The survey time lasted 5 
minutes and surveys were collected individually for each participant. 

Analysis of Results 

The analysis of the results employed a standard deviation as a standardized score to 
comparatively examine the values from Test 1, Test 2, basic academic ability, and 
computer experience. The amount of utterance was determined by converting the 
conversations recorded during the pair tests into text. The number of times students 
spoke was treated as the amount of utterances and the number of characters was 
treated as the utterance character count. The analysis of the pair results used in this 
study consists of the values that were calculated by subtracting the individual test 
scores (standard deviation) from each subject’s pair test scores (standard deviation), 
added according to pairs. 


Results and Interpretation 


Outline 

Looking at the results from the individual and pair practical tests 1 and 2, the pair 
tests (average standard deviation: 50.65 in 2008 and 51.62 in 2009) surpassed the 
scores from the individual test (average standard deviation: 49.34 in 2008 and 48.36 
in 2009) (p = 0.0015 for 2008 and p = 0.0001 for 2009). As an overall trend, this 
indicates that the task achievement level improves through pair problem solving. 
However, from an individual perspective, there was either no difference between the 
pair and individual results or the pair results were negative for close to 40% of 
students. On examining the relationship between pair and individual tests, trends were 
indicated in which pair problem solving had relatively less effect for students who 
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scored high in the individual test, while students who scored lower improved (Figure 
3). Previous research has also extrapolated that working in pairs is more effective for 
students with lower grades. 

40 


• y = -0.5658x + 29.643 

• R 2 = 0.3657 

30 



Individual Scores 

Figure 3 Individual scores and pair results 

Criteria for Pair Combination 

Table 1 Pair Results and Pair Combination Criteria 


_ . _ . . H High Pair Results) L Low 

Fair Combination 

Pair Results) 

■ t value 

P 

Criteria 

n 

m 

s.d. 

n 

m 

s.d. 

Basic Academic 








Q9 

Ability Difference 

8.85 

6.24 

36 

14.45 

8.29 

3.63 

0.001 ** 

Difference in PC 








92 

Experience 

8.88 

6.81 

36 

7.17 

6.86 

1.26 

0.211 


**p<.01 


As for the criteria for pair combination, analysis was conducted for two indicators 
suggested to be effective in the 2008 study, computer experience before university 
and basic academic ability (Table 1). Group H with a pair score above +10 and Group 
L with a score below -10 were selected in order to examine the characteristics of the 
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pair learning effect. A comparison of groups H and L indicated that the basic 
academic ability gap was small. The reason for this is that the gap in basic academic 
ability reflects the level of high school that students came from, the academic 
discipline, desire to leam, and class attitude. It is possible that these disparities affect 
the amount and quality of communication in pair testing. On the other hand, trends 
indicated that the gap in computer experience was greater in Group H and lesser in 
Group L, although the difference between the two was insignificant. The idea was that 
students with richer experience taught the students who lacked experience, which 
made the pairs more effective. However, the hypothesis is that for pairs with a lower 
computer experience, students would get stuck or need help in the same places, and 
although they consulted each other, they could not solve the problems. 

Table 2 Pair Results by Gender and Amount of Conversational Utterance 



Pair Gender 

Pair Results 

Amount of 
Conversational 
Utterance 


same 

0.85 

2138.3 

Male 


mixed 

6.3 

2114.8 


same 

1.68 

2200.1 

Female 


mixed 

4.3 

2127.4 


In addition to the two indicators—basic academic ability and computer experience 
before university—it was clear that gender was a factor in problem solving and 
performance. Males uttered less overall and male gender pairs were less effective, 
while mixed-gender pairs were more effective (Table 2). On the other hand, females 
overall were more vocal, although the result was that females vocalized more with 
same gender pairs as opposed to mixed-gender pairs. However, females achieved 
greater results with mixed-gender pairs as opposed to same gender pairs. Furthermore, 
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there was a high correlation between the amount of utterances and the pair results 
with females than with males. The outcome determined that mixed-gender pairs are 
more effective, followed by female pairs with male pairs being the least effective. 

From the above results, it can be concluded that the most effective pair combinations 
have a small gap in basic academic ability, a large gap in computer experience, and a 
partner of the opposite sex. 


Pair Learning Effect and the Amount of Utterances 

The vocal data (roughly 100 per year) collected during the pair test was converted 
into text. The conversation was analyzed by the amount of utterances and the 
character count of the utterance. 

There was a strong correlation (r = 0.98, y = 19.3x) between the amount of utterances 
and utterance character count. The average number of times students uttered during 
the 15 minute, 22 question (Q1-Q22) pair test was 106.0 and the average utterance 
character count was 2107. In other words, it was evident that there were 7 
conversational exchanges every minute and they spoke roughly 20 characters at a 
time. Moreover, depending on each pair, the utterance character count was 
disproportionate (highest was 4733 characters and lowest was 83 characters) and there 
was a large difference between the test results. Examination of the relationship 
between the overall utterance and pair results showed that vocal pairs were more 
effective (Figure 4, r = 0.42). 
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Looking at utterances for each question, there was more utterance for Q2 (insert a 
page number in the center of footer) in Test 2, which had a character count of 342, 
than Q9 (create an autoshape, and insert characters) in Test 1, which had a character 
count of 188. From these results, we can conclude that depending on the pair, there 
was a communication gap and a significant increase in utterances for problems with 
functions including a lot of steps or functions that were used less frequently during 
the class. 

Looking at the changes in utterances over time, utterances increased in the latter half 
of Test 1, which had a higher average score, and the utterance was particularly high 
for Q13-Q18. In contrast, Test 2 had higher utterances for Ql-Qll with significant 
reduction in the latter half. Furthermore, there was a difference between Test 1 and 
Test 2 for the pair learning effect by problem. 
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Figure 4 Relationship between pair results and amount of conversational utterance 
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Figure 5 Pair results by each question (•-Test 1 o-Test2) 


While it was more effective in questions Q13-Q18 for Test 1, Test 2 indicated 
negative values for Q16-Q21, which was lower than individual scores (Figure 5). 
Compared to Test 1, the difficulty level for Test 2 was slightly higher. This led 
students to spend more time communicating during the pair test, leaving less time for 
them to solve problems in the latter half of the test. 


As demonstrated above, the amount of utterances changed during problem solving for 
each pair depending on the difficulty level of the problem and their time management 
skills, suggesting that it impacted the positive effect of working in pairs. 


Learner Awareness for the Pair Test 

Judging from the results of the survey conducted after the pair test, a relationship 
between the effectiveness of pairs and a trend toward awareness of the pair test was 
considered. In 2008, the survey included 10 items in 2008, whereas it comprised 11 
items in 2009. 
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Table 3 Survey Items with Significant Differences from Pair Results 


Survey Item 

Pair Result 

n 

m 

P 

Judgement 


H 

174 

2.41 



Easier to solve as a pair than individually 

— 

— 

— 

- 0.0001 

** 


L 

86 

1.99 




H 

174 

2.54 



Consulted during the pair test 

—————— 

— 

- — 

- 0.0017 

** 


L 

86 

2.25 



A free conversation time before the pair test 

H 

98 

2.33 

- 0.0028 

** 

is necessary 

L 

42 

2.66 




H 

76 

0.28 



There was sufficient time for the pair test 




- 0.0053 

** 


L 

44 

0.07 




H 

174 

2.69 



Communication during the pair test was usef- 




- 0.0174 

* 


L 

86 

2.52 




Analysis of the significant difference between Group H, which was highly effective in 
terms of the pair learning effect, and Group L, which was less effective, was 
conducted with respect to these questionnaire items. The results of the common items 
from 2008 and 2009 were totaled together. 


First, Table 3 shows the items that pointed the significant differences. These results 
infer a willingness to solve problems cooperatively and communicate with each other, 
and whether or not they had sufficient time determined how effective pair learning 
was. As such, a positive attitude and increasing participation awareness of cooperative 
problem solving, expanding the ability to communicate, and improving time 
management skills are essential to promoting effective pairs. 
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We can interpret from the survey items (Table 4) where there was no significant 
difference between confidence in the class and students’ interest toward computers, 
and these items are unrelated to the effect. Free conversation time beforehand, the pair 
testing evaluation, and students’ interactions that were high across the board, are 
useful suggestions for setting up pair approach classes. 


Table 4 Survey Items with No Significant Difference with Pair Results 


Survey Item 

Pair Result 

n 

m 

p Judgement 


H 

76 

1.47 


Pair works were easy to understand 

— 

— 

— 

0.3193 


L 

44 

1.41 



H 

98 

2.72 


Free conversation time was sufficient 

— 

— 

— 

0.2808 


L 

42 

2.66 



H 

98 

2.65 


Pair work approach works well 

— 

— 

— 

0.2677 


L 

42 

2.60 



H 

76 

1.82 


Interest in computers 




0.2673 


L 

44 

1.72 



H 

98 

2.50 


Friendly interaction with partner 




0.2096 


L 

42 

2.60 
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Conclusion 

The results from the two-year experimental classes with pair testing provided the 
following findings within computer literacy education at university. 

1) Pair problem solving was higher than individual problem solving and it confirmed 
that pair task achievement was higher overall. On the other hand, from an 
individual perspective, working in pairs was ineffective or less effective for nearly 
40% students. 

2) The study inferred that the combination of criteria such as mixed-gender pairs with 
similar academic ability and differing computer experience was highly effective. 

3) The study discovered characteristics such as a greater discrepancy in the amount of 
utterances for certain pairs and remarkable increase in utterances for questions 
involving functions with more process steps or functions that were used less 
frequently in class. 

4) Amount of utterances changed depending on the difficulty level of the problem or 
their time management skills, indicating an impact on the effect of working in 
pairs. 

5) The study suggested that it is possible to improve pair learning results by 
improving students’ participation awareness and positive attitude toward 
cooperative learning, as well as improving their ability to communicate and time 
management skills. 

Further detailed analysis of the issues related to the pair learning approach will be 
conducted to resolve factors that affect the positive effect of pair learning. In addition, 
this study captured the pair learning effect through short-term experimental classes 
and consideration of further long-term application is necessary. 
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